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Nearly all science and engineering fields use search al¬ 
gorithms, which automatically explore a search space to 
find high-performing solutions: chemists search through the 
space of molecules to discover new drugs; engineers search 
for stronger, cheaper, safer designs, scientists search for 
models that best explain data, etc. The goal of search al¬ 
gorithms has traditionally been to return the single highest- 
performing solution in a search space. Here we describe a 
new, fundamentally different type of algorithm that is more 
useful because it provides a holistic view of how high- 

^_ * performing solutions are distributed throughout a search 

space. It creates a map of high-performing solutions at each 
^ point in a space defined by dimensions of variation that 
^ a user gets to choose. This Multi-dimensional Archive of 
O jPhenotvpic Elites (MAP-Elites) algorithm illuminates search 
spaces, allowing researchers to understand how interest¬ 
ing attributes of solutions combine to affect performance, 
O either positively or, equally of interest, negatively. For ex- 
^ ample, a drug company may wish to understand how per- 
formance changes as the size of molecules and their cost- 
HH to-produce vary. MAP-Elites produces a large diversity of 
^ high-performing, yet qualitatively different solutions, which 
^ can be more helpful than a single, high-performing solu- 
Q tion. Interestingly, because MAP-Elites explores more of the 
^search space, it also tends to find a better overall solution 
than state-of-the-art search algorithms. We demonstrate the 
benefits of this new algorithm in three different problem do- 
^ mains ranging from producing modular neural networks to 
^ designing simulated and real soft robots. Because MAP- 
Elites (1) illuminates the relationship between performance 
and dimensions of interest in solutions, (2) returns a set 
of high-performing, yet diverse solutions, and (3) improves 
,* the state-of-the-art for finding a single, best solution, it will 
^ catalyze advances throughout all science and engineering 
fields. 

t-H 

^ Author's Note: This paper is a preliminary draft of a paper that intro- 
. duces the MAP-Elites algorithm and explores its capabilities. Normally 
we would not post such an early draft with only preliminary experimen- 
I tal data, hut many people in the community have heard of MAP-Elites, 
^ are using it in their own papers, and have asked us for a paper that de¬ 
scribes it so that they can cite it, to help them implement MAP-Elites, 
and that describes the experiments we have already conducted with it. 
We thus want to share both the details of this algorithm and what we 
have learned about it from our preliminary experiments. All of the ex¬ 
periments in this paper will be redone before the final version of the paper 
is published, and the data are thus subject to change. 

1 Background and Motivation 

Every field of science and engineering makes use of search al¬ 
gorithms, also known as optimization algorithms, which seek to 
automatically find a high-quality solution or set of hi gh- quality 
solutions amongst a large space of possible solution£EI. Such 
algorithms often find solutions that outperform those designed 



Fig. 1. The MAP-Elites algorithm searches in a high-dimensional space 
to find the highest-performing solution at each point in a low-dimensional 
feature space, where the user gets to choose dimensions of variation of in¬ 
terest that define the low dimensional space. We call this type of algorithm 
an “illumination algorithm”, because it illuminates the fitness potential of 
each area of the feature space, including tradeoffs between performance 
and the features of interest. For example, MAP-Elites could search in the 
space of all possible robot designs (a very high dimensional space) to find 
the fastest robot (a performance criterion) for each combination of height 
and weight. 

by human ei^ineerJ^ they have designed antennas that NASA 
flew to spac^ found patentable electronic circuit designJ^, auto¬ 
mated scientific discover}^, and created artificial intelligence for 
robotJ^EH. 

Because of their widespread use, improving search al¬ 
gorithms provides substantial benefits for society 

Most search algorithms focus on finding one or a small set of 
high-quality solutions in a search space. What constitutes high- 
quality is determined by the user, who specifies one or a few ob¬ 
jectives that the solution should score high on. For example, a 
user may want solutions that are high-performing and low-cost, 
where each of those desiderata is quantifiably measured either by 
an equation or simulator. Traditional search algorithms include 
hill climbing, simulated annealing, evolutionary algorithms, gra¬ 
dient ascent/descent, Bawsian optimization, and multi-objective 
optimization algorithms^El The latter return a set of solutions that 
represent the best tradeoffs between objectiveJ^. 

A subset of optimization problems are challenging because 
they require searching for optima in a function or system that is 
either non-differentiable or cannot be expressed mathematically, 
typically because a physical system or a complex simulation is 
required. Such problems require "'black box" optimization algo¬ 
rithms, which search for high-performing solutions armed only 
with the ability to determine the performance of a solution, but 
without access to the evaluation function that determines that 
performance. On such problems, one cannot use optimization 
methods that require calculating the gradient of the function, 
such as gradient ascent/descent. 

A notorious challenge in black box optimization is the presence 
of local optima (also called local minima j®. A problem with most 
search algorithms of this class is that they try to follow a path that 
will lead to the best global solution by relying on the heuristic that 
random changes to good solutions lead to better solutions. This 
approach does not work for highly deceptive problems, however, 
because in such problems one has to cross low-performing valleys 
to find the global optima, or even just to find better optim^. 

Because evolutionary algorithms are one of the most success- 
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ful families of black-box search algorithmJ3El and because the 
work we build on comes from that community, here we adopt the 
language and metaphors of evolutionary computation. In that 
parlance, a solution is an organism or phenotype or individual, the 
organism is described by a genome or genotype, and the actions 
performed by that organism are the organism's behavior. The per¬ 
formance or quality of a solution is called its fitness, and the equa¬ 
tion, simulation, etc. that returns that fitness value is the fitness 
function. The way of stochastically producing new solutions is 
to take an existing solution and mutate its genome, meaning to 
change the genome in some random way, and or to produce a 
new solution descriptor by sampling portions of two parent de¬ 
scriptors, a process called crossover. Solutions that produce new 
offspring organisms are those that are selected, and such selection 
is typically biased towards solutions with higher fitnesP. 

To encourage a broad exploration of the search space, 
many modern evolutionary algorithms encourage diversity 
through a variety of different techniques, including increas¬ 
ing mutation rates w hen the rate of performance impr oye- 
ment stagnate^EE^^ e xplic itly selecting for genetic diversit}EE3 
or behavioral diversit}P“ or changing the structure of the 
populatiorPl Such diversity-promoting techniques often im¬ 
prove the quality of the solutions produced and the number of 
different types of solutions explored, but search algorithms still 
tend to converge to one or a f ew good solutions early and cease 
to make further progres^E^EH. 

An alternate idea proposed in recent years is to abandon the 
goal of improving performance altogether, and instead select 
only for diversity in the feature space (also called the behavior 
space): This algorithm, called Novelty Search, can perfor m bet- 
ter than performance-driven search on deceptive problemJ^^^. 
The user defines how to measure the distance between behav¬ 
iors, and then Novelty Search seeks to produce as many different 
behaviors as possible according to this distance metric. The al¬ 
gorithm stops when an individual in the population solves the 
objective (i.e. their performance is high enough). Because Nov¬ 
elty S earch does not work well with very large feature/behavioral 
space J2E11 there have been many prop osals for combining selec¬ 
tion for novelty and performance^^“ The main focus of these 
hybrid algorithms remains finding the single best individual that 
solves a task, or a set of individuals that represent the best possi¬ 
ble tradeoff between competing objectives. 

In the last few years, a few algorithms have been designed 
whose goal is not to return one individual that performs well on 
one objective, but a repertoire of in diyidua ls that each performs 
well on a different, related objectiv^^SES Along with research 
into behavioral diversity and Novelty Search, such repertoire¬ 
gathering algorithms inspire the algorithm we present in this pa¬ 
per. 

While the exploration of search spaces is at the center of many 
discussions in optimization, we rarely see these search spaces be¬ 
cause they are often too high-dimensional to be visualized. While 
the computer science literature offers plenty of options for dimen¬ 
sionality reduction and visualization of high-dimensional data 
such algorithms are "passive" in that they take a fixed data 
set and search for the best low-dimensional visualization of it. 
They do not tackle the issue of generating this data set. In other 
words, they do not explore a high-dimensional space in such a 
way as to reveal interesting properties about it to a user via a low¬ 
dimensional visualization. Such exploration algorithms are nec¬ 
essary when the entire search space is too large to be simply visu¬ 
alized by a dimensionality reduction algorithm, but instead must 
be actively explored to learn interesting facts about it. For exam¬ 
ple, to identify all the performance peaks in a large search space, 
we must actively search for them. It is not enough to sample mil¬ 
lions of solutions and plot them, for the same reason as random 
sampling is often not a good optimization algorithm: finding a 
fitness peak by chance is very unlikely for large search spaces 
(in most cases, the probability of finding the best possible fitness 


will decrease exponentially when the number of dimensions of 
the search space increases). 

Here we present a new algorithm that, given N dimensions of 
variation of interest chosen by the user, searches for the highest- 
performing solution at each point in the space defined by those 
dimensions (Fig. [^. These dimensions are discretized, with the 
granularity a function of available computational resources. Note 
that the search space can be high-dimensional, or even of infinite 
dimensions, but the feature space is low-dimensional by design. 
We call this algorithm the multi-dimensional archive of pheno¬ 
typic elites, or MAP-Elites. It was used and briefly described iiP, 
but this paper is the first to describe and explore its properties in 
detail. 

The benefits of MAP-Elites include the following: 

• Allowing users to create diversity in the dimensions of vari¬ 
ation they choose. 

• Illuminating the fitness potential of the entire feature space, 
not just the high-performing areas, revealing relationships 
between dimensions of interest and performance. 

• Improved optimization performance; the algorithm often 
finds a better solution than the current state-of-the-art search 
algorithms in complex search spaces because it explores 
more of the feature space, which helps it avoid local optima 
and thus find different, and often better, fitness peaks. 

• The search for a solution in any single cell is aided by the 
simultaneous search for solutions in other cells. This paral¬ 
lel search is beneficial because (1) it may be more likely to 
generate a solution for one cell by mutating a solution to 
a more distant cell, a phenomenon called "'goal switching" 
in a new paper that uses MAP-EliteJ^, or (2) if it is more 
likely to produce a solution to a cell by crossing over two 
solutions from other cells. If either reason is true, MAP- 
Elites should outperform a separate search conducted for 
each cell. There is evidence that supports this claim below, 
and this specific experiment was conducted in Nguyen et al. 
201pl which found that MAP-Elites does produce higher¬ 
performing solutions in each cell than separately searching 
for a high-performing solution in each of those cells. 

• Returning a large set of diverse, high-performing individ¬ 
uals embedded in a map that describes where they are lo¬ 
cated in the feature space, which can be used to create new 
types of algorithms or improve the performance of existing 
algorithms^ 

2 Optimization vs. iiiumination Aigorithms 

Optimization algorithms try to find the highest-performing solu¬ 
tion in a search space. Sometimes they are designed to return a 
set of high-performing solutions, where members in the set are 
also good on other objectives, and where the set represents the 
solution on the Pareto front of tradeoffs between performance 
and quality with respect to those other objectives. Optimization 
algorithms are not traditionally designed to report the highest- 
performing solution possible in an area of the feature space that 
cannot produce either the highest-performing solution overall, or 
a solution on the Pareto front. 

A different kind of algorithm, which we call illumination algo¬ 
rithms, are designed to return the highest-performing solution 
at each point in the feature space. They thus illuminate the fit¬ 
ness potential of each region of the feature space. In biological 
terms, they illuminate the phenotype-fitness mapl^. Any illumi¬ 
nation algorithm can also be used as an optimization algorithm, 
making illumination algorithms a superset of optimization algo¬ 
rithms. MAP-Elites is an illumination algorithm. It is inspired 
by two previous illumination algorithms. Novelty Search + Local 
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procedure MAP-Elites Algorithm (simple, 
('P^0,A^0) 
for iter = 1 ^ / do 
if iter < G then 

x' ^ random_solution() 
else 

X ^ random_selection(A’) 
x' ^ random_variation(x) 
b' ^feature_descriptor(x') 
p ^performance(x') 
if 'P(b') = 0 or 'P(b') < p' then 
V{h')^p' 

A(b') ^ x' 

return feature-performance map {V and A) 


DELAULT VERSION) 

> Create an empty, N-dimensional map of elites: {solutions A and their performances V} 

> Repeat for I iterations. 
> Initialize by generating G random solutions 

> All subsequent solutions are generated from elites in the map 
> Randomly select an elite x from the map A 
> Create x , a randomly modified copy of x (via mutation and/or crossover) 
> Simulate the candidate solution x' and record its feature descriptor h' 

> Record the performance p of x' 
> If the appropriate cell is empty or its occupants's performance is < p , then 
> store the performance of x' in the map of elites according to its feature descriptor h' 
> store the solution x' in the map of elites according to its feature descriptor h' 


Fig. 2. A pseudocode description of the simple, default version of MAP-Elites. 


Competition (NS-i-LCj^ and the Multi-Objective Landscape Ex¬ 
ploration algorithm (MOLEj^. All three are described below. 

3 Details of the MAP-Elites algorithm 

MAP-Elites is quite simple, both conceptually and to implement. 
Pseudocode of the algorithm is in Pig. Pirst, a user chooses a 
performance measure f{x) that evaluates a solution x. Por exam¬ 
ple, if searching for robot morphologies, the performance mea¬ 
sure could be how fast the robot is. Second, the user chooses N 
dimensions of variation of interest that define a feature space of 
interest to the user. Por robot morphologies, one dimension of in¬ 
terest could be how tall the robot is, another could be its weight, a 
third could be its energy consumption per meter moved, etc. An 
alternate example could be searching for chess programs, where 
the performance measure is the win percentage, and the dimen¬ 
sions of variation could be the aggressiveness of play, the speed 
with which moves are selected, etc. A further example is evolv¬ 
ing drug molecules, where performance could be a drug's effi¬ 
cacy and dimensions of variation could be the size of molecules, 
the cost to produce them, their perishability, etc. 

Each dimension of variation is discretized based on user prefer¬ 
ence or available computational resources. This granularity could 
be manually specified or automatically tuned to the available re¬ 
sources, including starting with a coarse discretization and then 
increasing the granularity as time and computation allow. 

Given a particular discretization, MAP-Elites will search for the 
highest performing solution for each cell in the A-dimensional 
feature space. Por example, MAP-Elites will search for the fastest 
robot that is tall, heavy, and efficient; the fastest robot that is tall, 
heavy, and inefficient, the fastest robot that is tall, light, and effi¬ 
cient, etc. 

The search is conducted in the search space, which is the space 
of all possible values of x, where x is a description of a candidate 
solution. In our example, the search space contains all possible 
descriptions of robot morphologies (note that we must search in 
the space of descriptions of robot morphologies; it is not possible 
to search directly in the space of robot morphologies or directly 
in the feature space). We call the x descriptor a genome or genotype 
and the robot morphology the phenotype, or px. We have already 
mentioned that a function f{x), called a. fitness function, returns 
the performance of each x. A feature (a.k.a. behavior) function 
b{x) must also exist or be defined that, for each x, determines x's 
value in each of the N feature dimensions. In other words, b{x) 
returns bx, which is an A-dimensional vector describing x's fea¬ 
tures. In our example, the first dimension of bx is the robot's 
height, the second dimension is its weight, and the third is its en¬ 
ergy consumption per meter moved, etc. Some elements of the 
feature vector may be directly measured in the phenotype (e.g. 
height, weight), but others (e.g. energy consumption) require 


measuring the behavior of the phenotype while it performs, ei¬ 
ther in simulation or reality. 

Note that there may be many levels of indirection between x 
and bx. With direct encoding, each element in the ge nome specifies 
an independent component of the phenotypd3®l. In that case, it 
is straightforward to map genotypes into phenotypes, and then 
measure performance and features (evaluating the phenotype in 
a simulator or the real world if necessary). An extra level of indi¬ 
rection can occur with indirect encoding, also known as generative 
or developmental encoding, in which information in the genome 
can be reused to affect many parts of the phenotype (also called 
pleiotropy); such encodings have been shown to improve regular¬ 
ity, performance, and evolvabilit}EEEE3i®E3. In other words, a 
complex process can exist that maps genome x ^ to phenotype 
Px to features bx and performance fx. 

MAP-Elites starts by randomly generating G genomes and de¬ 
termining the performance and features of each. In a random or¬ 
der, those genomes are placed into the cells to which they belong 
in the feature space (if multiple genomes map to the same cell, the 
highest-performing one per cell is retained). At that point the al¬ 
gorithm is initialized, and the following steps are repeated until 
a termination criterion is reached. (1) A cell in the map is ran¬ 
domly chosen and the genome in that cell produces an offspring 
via mutation and/or crossover. (2) The features and performance 
of that offspring are determined, and the offspring is placed in 
the cell if the cell is empty or if the offspring is higher-performing 
than the current occupant of the cell, in which case that occupant 
is discarded. 

The termination criterion can be many things, such as if a set 
amount of time expires, a fixed amount of computational re¬ 
sources are consumed, or some property of the archive is pro¬ 
duced. Examples of the latter could include a certain percentage 
of the map cells being filled, average fitness in the map reaching 
a specific level, or n solutions to a problem being discovered. 

One can consider the archive, which is the set of descriptors in 
all the cells, as the traditional population in an evolutionary algo¬ 
rithm. The difference is that in MAP-Elites each member of the 
population is by definition diverse, at least according to the di¬ 
mensions of the feature space. 

The above description, for which pseudocode is provided in 
FigH is the default way to implement MAP-Elites. To encour¬ 
age a more uniform exploration of the space at a coarse resolu¬ 
tion, and then a more fine-grained search afterwards, we created 
a hierarchical version that starts with larger cells in the feature 
space that are then subdivided into smaller cells during search af¬ 
ter predetermined numbers of evaluations have been performed 
(Methods). We further parallelized this algorithm to run on clus¬ 
ters of networked computers, by farming out batches of evalua¬ 
tions to slave nodes, instead of performing each evaluation seri¬ 
ally (Methods). Section [^contains ideas for additional, alternate 
possible variants of MAP-Elites. 
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There are two things to note about MAP-Elites: 

• It is not guaranteed that all cells in the feature space will be 
filled, for two reasons. (1) There may be no genome x that 
maps to a particular cell in the feature space. For example, 
it may be impossible to have a robot of a certain height and 
weight due to physical laws. (2) The search algorithm may 
fail to produce a genome that maps to a cell in the feature 
space, even if such a genome exists. 

• There are many genotypes that can be mapped to the same 
cell in the feature space, perhaps an infinite number. For ex¬ 
ample, there are many robot blueprints that produce a robot 
with the same height, weight, and energy consumption. For 
that reason, and because it is not known a priori which 
genomes will map to which cells, it is not possible to search 
directly in the feature space. Recall that there is, even with 
direct encodings, and especially with indirect encodings, a 
complex mapping from genome x to the feature vector bx. If 
it is possible in a given problem to directly take steps in the 
feature space, then MAP-Elites is unnecessary because one 
could simply perform exhaustive search in the feature space. 
One can think of MAP-Elites as a way of trying to perform 
such an exhaustive search in the feature space, but with the 
additional challenge of trying to find the highest-performing 
solution for each cell in that feature space. 

4 Differences between MAP-Elites and previous, 
related algorithms 

In 2011, Lehman and Stanle}?^ note that combining a selective 
pressure for feature diversity with one performance objective that 
all of the different types of phenotypes compete on is akin to hav¬ 
ing butterflies and bacteria compete with bears and badgers on 
one performance criterion (e.g. speed). Doing so is unhelpful 
for producing the fastest of each type of creature, given the dif¬ 
ferent speed scales these creatures exhibit. Instead, Lehman and 
Stanley propose encouraging diversity in the feature space, but 
having each organism compete on performance only with other 
organisms that are near it in the feature space, an algorithm they 
call Novelty Search + Local Competition (NS+LCj^. NS+LC accom¬ 
plishes these goals via a multi-objective algorithm with two ob¬ 
jectives: (1) maximizing an organism's performance relative to its 
closest 15 neighbors (i.e. local competition, but note that these rel¬ 
ative scores are then entered into a global competition, the impli¬ 
cations of which are discussed below), and (2) maximizing a nov¬ 
elty objective, which rewards organisms the further they are in 
feature space from their 15 closest neighbors. Whereas normally 
evolutionary algorithms do not produce much diversity within 
one run, but instead have to perform multiple, independent runs 
to showcase diversit}^^, NS+LC produces a substantial amount of 
different types of high performing creatures within one evolving 
populatiorW 

NS+LC inspired us to create two algorithms that also seek to 
find the highest performing solution at each point in a feature 
space. The first was the Multi-Objective Landscape Exploration 
(MOLE) algorithrrPand the second is MAP-Elites, the algorithm 
presented in this paper. MOLE has two objectives: the first is 
performance, and the second for each organism to be as far from 
other organisms as possible, where distance is measured in a fea¬ 
ture space that a user specifies. 

Both NS+LC and MOLE have similar goals to MAP-Elites: they 
search for the highest-performing solution at each point in a fea¬ 
ture space. However, both are more complicated and, as will be 
shown in the results section, do not perform as well as MAP- 
Elites empirically. 

Specific differences between MAP-Elites and NS+LC include: 

• Novelty Search needs to compute the feature distance to ev¬ 
ery other organism each generation; such nearest neighbor 


calculations are 0(n log (n)j^ each generation. MAP-Elites 
only needs to look up the current occupant of the cell, which 
is 0(1). 

• Novelty Search contains both a current population and an 
archive of previous solutions that serves as a memory of 
which points in the feature space have been visited. Main¬ 
taining both a population and an archive requires many ad¬ 
ditional parameters that have to be chosen carefully or per¬ 
formance can be detrimentally affectecP^. 

• Given that Novelty Search rewards individuals that are dis¬ 
tant from each other in the feature space, having only a pop¬ 
ulation would lead to "cycling", a phenomenon where the 
population moves from one area of the feature space to a new 
area and back again, without any memory of where it has al¬ 
ready explored. The archive in NS+LC limits, but does not 
eliminate, this phenomenon. MAP-Elites does away with the 
archive vs. population distinction by having only an archive. 
It thus avoids cycling and is always simultaneously focused 
on expanding into new niches (until there are none left) and 
improving the performance of existing niches. 

It is thus quite easy to intuit what the selection pressure for 
MAP-Elites is over time. In contrast, the selection pressures 
for Novelty Search are more dynamic and thus harder to un¬ 
derstand, even for Novelty Search variants that have only an 
archive and no populatiorP^. For example, it is hard to pre¬ 
dict how much search will be focused in each area of the fea¬ 
ture space, because a relatively sparse area during one era of 
the search can become relatively crowded later on, and vice 
versa. 

The dynamics of NS+LC are even more dynamic, complex, 
and unpredictable. One thing to keep in mind is that, while 
the performance of solutions in NS+LC is only judged ver¬ 
sus neighbors, these relative performance scores are then 
competed globally within the (relative) performance objec¬ 
tive. Overall, therefore, NS+LC biases search towards under¬ 
explored areas of the feature space (taking into account 
the archive and the population), areas of the search space 
with the highest relative performance, and tradeoffs between 
these two objectives. An organism in an area that is bet¬ 
ter than its neighbors, but where this gap is not as large 
as elsewhere, will not be explored as often unless or until 
that larger performance gap elsewhere is reduced. The fo¬ 
cus of the (relative) performance objective is thus complex 
and ever-changing. The diversity objective is also complex 
and dynamic, because NS+LC does not only store one so¬ 
lution per cell. Many solutions can pile up in one area of 
the space, creating a pressure to explore under-explored ar¬ 
eas until those areas are more explored relative to the initial 
area, creating a pressure to return to the initial area, and so 
on. 

For both objectives, thus, it is hard to intuit both the dy¬ 
namics themselves and what effects these dynamics have on 
search. MAP-Elites, in contrast, produces offspring by uni¬ 
formly sampling from the existing archive, such that the only 
thing that changes over time is the number of cells that are 
filled and their performance. MAP-Elites thus embodies the 
main principle of illumination algorithms, which is to search 
for the highest-performing solution at each point of the fea¬ 
ture space, in a more simple, intuitive, and predictable way. 

• In the default version of MAP-Elites, organisms only com¬ 
pete with the organism (the current occupant) in their cell, so 
the range of features they compete with is fixed. In Novelty 
Search and NS+LC, organisms compete with their nearest 
neighbors in the feature space. Especially at the beginning of 
the run before the archive fills up, that might mean that or¬ 
ganisms are competing with others that have very different 
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features, which is contrary to the spirit of local competition 
in the feature space. 

Specific differences between MAP-Elites and MOLE include: 

• MOLE features one global performance competition (via the 
performance objective). Thus, a few high-performing indi¬ 
viduals will dominate this objective, making it hard to rec¬ 
ognize and keep a slightly better performing solution in a 
low- or medium-performing region of the space. MAP-Elites 
is better at recognizing and keeping any improvement to fit¬ 
ness in any region of the space, no matter how the perfor¬ 
mance of that cell compares to other cells. As an example of 
when MOLE might fail to reward an important innovation, 
imagine a new solution in a medium-performing, densely 
packed region of the space, that is higher-performing than 
anything previously found in that cell. This new solution, 
which represents the best performance yet found in that 
cell, would not be selected for because it is neither high- 
performing versus other organisms in the population, nor 
would it be kept because it is diverse. Thus, the organism 
does not perform well in either of the MOLE objectives, yet 
it is precisely what we truly want: the highest performing 
individual found so far in that area of the feature space. 

• Like Novelty Search, the diversity objective in MOLE has un¬ 
stable temporal dynamics. The population may rush to a rel¬ 
atively unexplored area, fill it up, then rush off to a new rel¬ 
atively unexplored area, and then rush back to the original 
area. It does not evenly search for improvements to all areas 
of the map simultaneously. 

5 Criteria for Measuring the Aigorithms 

There are many different ways to quantify the quality of illumi¬ 
nation algorithms and optimization algorithms. In this paper, we 
evaluate algorithms on the following quantifiable measures: 

• Global Performance: Eor each run, the single highest- 
performing solution found by that algorithm anywhere in 
the search space divided by the highest performance possi¬ 
ble in that domain. If it is not known what the maximum 
theoretical performance is, as is the case for all of our do¬ 
mains, it can be estimated by dividing by the highest perfor¬ 
mance found by any algorithm in any run. This measure is 
the traditional, most common way to evaluate optimization 
algorithms. One can also measure whether any illumination 
algorithm also performs well on this measurement. Both the 
ideal optimization algorithm and the ideal illumination al¬ 
gorithm are expected to perform perfectly on this measure. 

• Global reliability: Eor each run, the average across all cells 
of the highest-performing solution the algorithm found for 
each cell (0 if it did not produce a solution in that cell) di¬ 
vided by the best known performance for that cell as found 
by any run of any algorithm. Cells for which no solution was 
found by any run of any algorithm are not included in the 
calculation (to avoid dividing by zero, and because it may 
not be possible to fill such cells and algorithms thus should 
not be penalized for not doing so). Section [9.4.1 [ provides the 
formal equation. 

This measure assesses how reliable an algorithm is at find¬ 
ing the highest-performing solution for each cell in the map. 
It is the most important measure we want an illumination 
algorithm to perform well on, and the ideal illumination al¬ 
gorithm would perform perfectly on it. There is no reason 
to expect pure optimization algorithms, even ideal ones, to 
perform well on this criterion. 


• Precision (opt-in reliability): Eor each run, if (and only if) 
a run creates a solution in a cell, the average across all such 
cells of the highest performing solution produced for that cell 
divided by the highest performing solution any algorithm 
found for that cell. Section [9.4.2| provides the formal equa¬ 
tion. 

This metric measures a different notion of reliability, which 
is the trust we can have that, if an algorithm returns a solu¬ 
tion in a cell, that solution will be high-performing relative to 
what is possible for that cell. To anthropomorphize, the al¬ 
gorithm gets to opt-in which cells it wishes to fill and thus 
be measured on. The ideal illumination algorithm would 
have a perfect score of 1 for this criterion. Optimization al¬ 
gorithms should fare better on this criterion than global reli¬ 
ability, because they will tend to explore only a few areas of 
the feature space, but should produce high-performing so¬ 
lutions in many cells in those areas. Note, however, that if 
an optimization algorithm starts in a low-performing region 
of the feature space and moves to a neighboring region, it is 
expected that its relative performance in the cells it started 
in will stay low, as optimization algorithms are not asked to 
improve performance in those cells. Thus, even ideal opti¬ 
mization algorithms are not expected to perform perfectly 
on this criterion on average, although they may do so once 
in a while. 

• Coverage: Measures how many cells of the feature space a 
run of an algorithm is able to fill of the total number that are 
possible to fill. The mathematical details are specified in sec- 
tion |9.4.3] This measure does not include the performance of 
the solutions in the filled cells. The ideal illumination algo¬ 
rithm would score perfectly on this metric. The ideal opti¬ 
mization algorithm is not expected to perform well on this 
criterion. 

6 Experiments and Results 

We evaluated MAP-Elites in three different search spaces: neu¬ 
ral networks, simulated soft robot morphologies, and a real, soft 
robotic arm. The neural network search space is interesting be¬ 
cause evaluations are fast, allowing us to draw high-resolution 
feature-space maps for a high-dimensional search space. The 
experiments with both simulated and real soft robot are inter¬ 
esting because soft robots are important, new design spaces 
where traditional design and control methods do not work well, 
if at all. Thus, we need advanced search algorithms to find 
high-performing designs. The first two search spaces (neu¬ 
ral networks and simulated soft robots) are extremely high¬ 
dimensional, demonstrating the ability of MAP-Elites to cre¬ 
ate low-dimensional feature maps from high-dimensional search 
spaces. The third, involving the soft robot arm, involves eval¬ 
uations that are performed directly on a real robot because the 
soft properties of the robot are too complex to simulate. This do¬ 
main demonstrates that MAP-Elites is also effective even in a low¬ 
dimensional, challenging, real-world problem. 

6.1 Search space 1: neural networks 

This problem domain is identical to one from Clune et al. 201^, 
which itself is based on the domain from Kashtan and Alon 
200^^. The following explanation of the domain is adapted from 
Clune et al. 20 

The problem involves a neural network that receives stimuli 
from an eight-pixel retina. Patterns shown on the retina's left and 
right halves may each contain an object (i.e. a pattern of interest). 
Networks have to answer whethe r an object is present on both 
the left and right sides of the retinaPSEl Each network iteratively 
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sees all possible 256 input patterns and answers true (> 0) or false 
(< 0). Its performance is the percentage of correct answers. 

Because it has been shown that minimizing connection costs 
promotes the evolution of modularit}@, it is interesting to vi¬ 
sualize the relationship between network connection costs and 
modularity. To do so, we can create a 2D feature space where the 
first feature dimension {x axis) is connection cost (the sum of the 
squared length of the connections in a networlP^, and the sec¬ 
ond feature dimension is network modularity (computed using 
an efficient approximation of Newman's modularity scor^^. The 
resolution of the map is 512 x 512; the map is filled by the hierar¬ 
chical version of MAP-Elites with 10,000 evaluations (Methods). 

For this domain, we compare MAP-Elites to three other algo¬ 
rithms: (1) a traditional, single-objective evolutionary algorithm 
with selection for performance only, which thus does not explic¬ 
itly seek diversity in either feature dimension, (2) novelty search 
with local competition (NS+LC) which is described above, 
where novelty is measured in the same 2D feature space, and 
(3) to random sampling. For these three control experiments, we 
record all the candidate solutions evaluated by the algorithm and 
then keep the best one found per cell in the feature space (i.e. 
the elite performer for each cell), and report and plot these data. 
Each treatment is allocated the same number of fitness evalua¬ 
tions (Methods). For each treatment, 20 independent runs are per¬ 
formed, meaning 20 independent replicates that each start with a 
different random number seed and thus have different stochastic 
events. 

The results reveal that MAP-Elites scores significantly higher 
(p < 1x10“^) than the three control algorithms on all four criteria 
described in section]^ global performance, global reliability, pre¬ 
cision, and coverage (Fig. ^ Top). Qualitatively, the difference in 
MAP-Elites vs. the controls is apparent in typical, example maps 
produced by each treatment (Fig. Bottom). Overall, MAP-Elites 
finds solutions that are more diverse and high-performing than 
traditional optimization algorithms (here called the "traditional 
FA"), novelty search with local competition, and random sam- 
pling. 

It is surprising that, even when looking only at the best perfor¬ 
mance overall (global performance), MAP-Elites outperforms the 
traditional EA, which focuses explicitly on finding the single best¬ 
performing individual in the search space. That is likely because 
the retina problem is deceptiv^and this traditional evolutionary 
algorithm has no pressure for diversity, which is known to help 
with deceptiorP. 

While MAP-Elites significantly outperforms all controls on 
both reliability and precision (opt-in reliability), the gap is much 
narrower for precision, as is to be expected. In terms of coverage, 
random sampling was the second best of the algorithms in our 
study. MAP-Elites likely outperforms it in this regard because 
mutations to members of a diverse population are more likely 
to fill new cells versus randomly generating genomes. That is 
especially true if cells are more likely to be filled by mutating a 
nearby cell than by randomly sampling from the space of all pos¬ 
sible genotypes. Imagine, for example, that most randomly sam¬ 
pled genotypes are in the center of a map. In that case, it would 
be unlikely to produce an organism in a corner by random sam¬ 
pling. In contrast, MAP-Elites could slowly accumulate organ¬ 
isms in cells closer and closer to the corner, making it more likely 
to eventually fill that corner. Random sampling likely produces 
more coverage than the traditional EA because the latter tends to 
allocate new individuals as offspring of the highest-performing 
organisms found so far, focusing search narrowly at the expense 
of exploring the feature space. It is not clear why random sam¬ 
pling produced more coverage than NS+LC, although this result 
needs to be tested across a wider range of parameters before its 
robustness is known. 

We can also report anecdotally that MAP-Elites performs much 
better in this domain than the MOLE algorithm, which was pre¬ 
viously applied to this same domain and feature spac^. For this 


early draft of the paper we do not yet have data to share because 
the MOLE runs in that paper were at a lower resolution; we will 
add a fair comparison of MOLE to MAP-Elites in a future draft of 
this paper. We can report that the MOLE figures from Clune et al. 
2013 required merging data from many (specifically, 30) different 
runs of MOLE, meaning that across many MOLE runs we took 
the highest-performing network found in each cell. The variance 
in these MOLE runs was high, such that many of the runs did not 
find high-performing networks in large regions of the space; we 
thus were only able to get a good picture of the fitness potential of 
each region by taking data from many different runs. That high 
variance means that any individual MOLE run did not produce 
a reliable, consistent, true picture of the fitness potential of each 
region of the space; such a picture only came into view with a 
tremendous amount of computation spent on many MOLE runs. 
In contrast, each individual MAP-Elites run produces a consistent 
picture that looks similar to the result of merging many MOLE 
runs. There is still variance between MAP-Elites runs, but it is 
much smaller, meaning that each run of the algorithm is more 
reliable. 

We next investigated the assumption that elites are found by 
mutating genomes nearby in the feature space, and found that 
this assumption is largely true (Fig. Left). Most organisms de¬ 
scend from nearby organisms, whether close neighbors, nearby 
neighbors, or more distant neighbors within the same region of 
the space. None of the organisms we randomly sampled were 
produced by a parent more than halfway across the feature map. 
That said, many high-performing elites do descend, not from 
immediate neighbors, but from a high-performing neighbor a 
medium distance away. That fact shows that purely local search, 
which likely concentrates on one area of the feature space, may 
not be the best way to discover high-performing solutions, and 
suggests that one reason MAP-Elites is able to find so many high- 
performing solutions is because collecting a large reservoir of 
diverse, high-performing solutions makes it more likely to find 
new, different, high-performing solutions. 

Looking at the direct parents of elites suggests that a relatively 
local, but overlapping, search is taking place in each region of the 
map. However, looking at the entire lineage of four randomly 
chosen elites reveals that lineages frequently traverse long paths 
through many different regions of the map (Fig. right). These 
lineages further underscore the benefit of simultaneously search¬ 
ing for high-performing organisms at each point in the map: do¬ 
ing so may provide stepping stones to high-performing solutions 
in a region that may not have been discovered had search been 
trying to increase performance by searching only in that region. 
This result was replicated in a recent study in a different domain 
that investigated this issue with MAP-Elites in more depthP^ 

6.2 Simulated soft, locomoting robot morphologies 

Soft robots are made of soft, deformable materials; they open up 
new design spaces, allowing the creatio n of r obots that can per¬ 
form tasks that traditional robots cannoP^^. For example, they 
can adapt their shape to their environment, which is useful in re¬ 
stricted spaces like pipelines, caves, and blood arteries. They are 
also safer to have around human^. However, they are harder 
to design because their components have many more non-linear 
degrees of freedorrP^El. 

It has previously been shown that an evolutionary algorithm 
with a modern, generative encoding (explained below) can pro¬ 
duce a diversity of soft robots morphologies that move in dif¬ 
ferent wayJ^. However, the diversity of morphologies shown in 
that paper and its accompanying video (https : //youtu .be/ 
z9ptOeByLA4) came from different runs of evolution. Within each 
run, most of the morphologies were similar. As is typical with 
evolutionary algorithms, in each run the search found a local op¬ 
timum and became stuck on it, spending most of the time explor¬ 
ing the similar designs on that peak. 
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(e) MAP-Elites 


Fig. 3. MAP-Elites produces significantly higher-performing and more diverse solutions than control algorithms. Top: MAP-Elites significantly out¬ 
performs controls on global performance (finding the single highest-performing solution), reliability (average performance across all fillable cells), 
precision (average performance only of cells filled by the algorithm), and coverage (the number of cells filled). All of the metrics are normalized. 
Section [^explains these metrics in more detail. In the box plot for each metric, the black line shows the median. Bottom: Example maps produced 
by a sin^e run (the first one) of each treatment. As described in Clune et al. 201 d^, the x-axis is connection cost, the y-axis is modularity, and heat 
map colors indicate normalized performance. These maps show that MAP-Elites illuminates more of the feature space, revealing the fitness potential 
of each area. 


The morphologies evolved in Cheney et al.^ also rarely in¬ 
cluded one of the four materials available, a stiff (dark blue) ma¬ 
terial analogous to bone. The authors (one of which is the last 
author on this paper) tried many different parameters and envi¬ 
ronmental challenges to encourage the optimization algorithm to 
use more of this material, but it rarely did. One could, of course, 
explicitly include a term in the fitness function to reward the in¬ 
clusion of this material, but that may cause evolution to over in¬ 
vest in it, and it is hard to know ahead of time how much material 
to encourage the inclusion of to produce interesting, functional 
designs. The ideal would be to see the highest-performing crea¬ 
ture at each level of bone use, and thus learn how the use of bone 
affects both fitness and morphology design. That is exactly what 
MAP-Elites is designed for. The authors of Cheney et al. 2013 


were also interested in morphologies of different sizes, which can 
also be added as a different dimension of variation to be explored 
by MAP-Elites. 

Here we test whether MAP-Elites can address the issues raised 
in the two previous paragraphs. Specifically, we test (1) whether 
MAP-Elites can produce a large diversity of morphologies within 
one run and (2) whether it can produce high-performing mor¬ 
phologies for a range of levels of bone use and body size, and 
combinations thereof. 

We adopt the same domain as Cheney et al. 201^ by evolving 
multi-material, soft robots in the Voxcad simulatoi^. Robots are 
specified in a space of 10 x 10 x 10 voxels, where each voxel is ei¬ 
ther empty or filled with one of four kinds of material: bone (dark 
blue, stiff), soft support tissue (light blue, deformable), muscles 
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Fig. 4. Most elites are found by mutating a parent genome that was nearby in the feature space, but the entire lineages of example elites reveals 
search paths that traverse large distances through the feature space. The data in these plots are from the neural network domain. As in Fig.[^ 
and Fig. 3 of Clune et al.EH, the a^-axis is connection cost and the y-axis is modularity. Left: For a random subset of elites from the neural networK 
domain, we draw an arrow pointing at that elite that starts in the location of the parent that produced that elite. If there were no correlation between the 
location of an elite and its parent, there would be far more long arrows. Most elites are produced from parents within a range of distances in a nearby 
region (approximately 0.2 or less). The color of the beginning of each arrow denotes the performance of the parent, and the color toward the tip of 
the arrow denotes the performance of the elite. Note that many high-performing elites descend from other high-performing elites, but often not from 
direct neighboring cells. These data suggest that collecting high-performing elites in many different locations helps discover high-performing elites in 
new locations, which is likely why MAP-Elites is able to find so many different, high-performing solutions. Right: Example lineages tracing all of the 
descendants of four randomly selected final elites. For each of the four elites, a dashed line of a different color (green, orange, blue, or purple) starts at 
its randomly generated, generation 0 ancestor (red circle), which interestingly is the same for all four elites. Note that the colors and paths are harder to 
differentiate when the different lineages overlap. Each dashed line passes through the location in the feature space of each ancestor along the lineage 
of that elite and terminates at that elite’s location in the feature space. The color of arrows along each lineage denote the performance of the parent 
that was located at the tail end of the arrow and produced the offspring at arrowhead. The main conclusion is that the stepping stones that lead to a 
high-performing elite at a particular location in the feature space are distributed throughout the feature space, suggesting that MAP-Elites’ strategy of 
simultaneously rewarding high-performing organisms at each point in the space may help discover high-performing solutions in very different regions. 


that contract and expand in phase (green, cyclical volumetric ac¬ 
tuation of 20%), and muscles that contract and expand in opposite 
phase (red, counter-cyclical volumetric actuation of 20%). 

The material of each voxel is encoded with a compositional 
pattern-producing network (CPPN)^, an encoding based on de¬ 
velopmental biology that ca uses robot p henotypes to be more 
regular and high-performinJ33SE3^33. CPPNs are similar to 
neural networks, but with evolvable activation functions (in this 
paper, the functions can be sine, sigmoid, Gaussian, and linear) 
that allow the network to create geometric patterns in the pheno¬ 
types they encode. Because these activation functions are regu¬ 
lar mathematical functions, the phenotypes produced by CPPNs 
tend to be regular (e.g. a Gaussian function can create symme¬ 
try and a sine function can create repetition). CPPN networks are 
genomes that are run iteratively for each voxel in the workspace 
to determine whether that voxel is empty or full and, if full, which 
type of material is present. Specifically, for each voxel, the Carte¬ 
sian {x, y, and z) coordinates of the voxel and its distance from the 
center {d) are provided as inputs to the CPPN, and one CPPN out¬ 
put specifies whether a voxel is empty. If the voxel is not empty, 
the maximum value of an additional four outputs (one per ma¬ 
terial type) determines the type of material for that voxel. This 
method of separating the presence of a phenotypic component 
and its parameters into sepa rate CPPN outputs has been shown 
to improve performance^^El if there are multiple disconnected 
voxel patches, only the most central patch is considered as the 
robot morphology. A lengthier explanation of CPPNs and how 
they specify the voxels of the soft robots in this domain can be 
found in Cheney et al. 20from which some text in this de¬ 
scription of methods was derived. 

While the soft robot morphologies are indirectly encoded by 
CPPNs, the CPPN networks themselves are directly encoded and 
evolved according to the principle s of the NEAT algorithrrP^, as is 
customary for CPPNj33Si3^^“ Here, the NEAT principles are 


implemented in the Sferes^ evolutionary platform, which has 
some departures from the original NEAT algorithm. Specifically, 
our direct encoding does not include crossover or genetic diver¬ 
sity via speciation. See Mouret and Doncieux 2012^ for a more 
detailed description of the Sferes version of NEAT. 

Performance for these soft robots is defined as the distance cov¬ 
ered in 10 simulated seconds. The first (a:-axis) dimension of the 
feature space is the percentage of voxels that are the stiff bone 
(dark blue) material. The second feature-space dimension is the 
percentage of voxels filled. The resolution of the map is 128 x 128. 
We launched 10 runs for each treatment, but some had not com¬ 
pleted in time to be included in this draft of the paper. We thus 
include data only from runs that finished in our plots and statisti¬ 
cal analyses (7 for the EA treatment, 5 for the EA-i-Diversity treat¬ 
ment, and 8 for the MAP-Elites treatment). In later drafts of this 
paper we will report on a complete set of finished experiments, 
which will also have a larger and consistent number of runs per 
treatment. 

Our two control algorithms are irnpl emented in NSGA-II and 
have been used in previous studiesl^^M (l) a single-objective evo¬ 
lutionary algorithm optimizing performance only, which we refer 
to as the 'Traditional EA" or just "EA" for short, and (2) a two- 
objective evolutionary algorithm that optimizes performance and 
diversity, which we call EA-i-D. Diversity is measured for each 
individual as the average distance in the feature space to every 
other individual. Both control treatments performed the same 
number of evaluations as MAP-Elites. 

In this domain, MAP-Elites does a far better job than the con¬ 
trols of revealing the fitness potential of each area of the fea¬ 
ture space, which is the goal of illumination algorithms (Eig.|^. 
It has significantly higher reliability and coverage (p < 0.002), 
and example maps highlight the tremendous difference in terms 
of exploring the feature space between the MAP-Elites illumina¬ 
tion algorithm and the two control optimization algorithms, even 
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though one has a diversity pressure. 

In terms of global performance, while MAP-Elites has a higher 
median value, there is no significant difference between it and the 
other treatments {p > 0.05). If one cared only about finding a sin¬ 
gle, high-performing solution, then there would thus be no sta¬ 
tistical difference between MAP-Elites and the two optimization 
algorithm controls. However, if one wanted a variety of different, 
high-performing solutions, MAP-Elites produces far more. 

MAP-Elites is significantly worse at precision than the two con¬ 
trol algorithms {p < 0.01). This result is likely explained by the 
fact that the control algorithms allocate all of their evaluations to 
very few cells, and thus find good solutions for those cells. In 
contrast, MAP-Elites has to distribute its evaluations across or¬ 
ders of magnitude more cells, making it hard to always find a 
high-performing solution in each cell. Note that MAP-Elites is 
usually competing against itself in this regard: because there is so 
little exploration by the control algorithms, they rarely produce 
the highest-performing solution across all runs of all treatments 
for a particular cell. Those instead tend to come from MAP-Elites. 
Thus, most low precision scores for MAP-Elites come when one 
run of MAP-Elites does not find as high-performing a solution in 
a cell as another run of MAP-Elites. We hypothesize that if each 
run of MAP-Elites were given more evaluations (i.e. run longer), 
it would catch up to, if not surpass, the controls in precision. That 
is a beneficial, and rare, property for an evolutionary algorithm 
to have: that it can benefit from additional computation because 
it does not get stuck on local optima and cease to innovate. 

There is variance in the maps produced by independent runs 
of MAP-Elites. That reflects the fact that it is a stochastic search 
algorithm with historical contingency. The perfect illumination 
algorithm would always find the highest-performing solution at 
each point in the map, and thus have no between-run variance. 
However, while there are differences between the maps of differ¬ 
ent runs, they largely reveal the same overall pattern (Eigs.j^and 

E}. 

By looking at pictures and videos of the elites in the final map 
of individual runs, we observed that MAP-Elites does indeed pro¬ 
duce smooth changes across the chosen dimensions of variation 
(Eig. [6). Consider the column of examples from the right side 
of Eig.l^ Top, where the percent of voxels filled is roughly 75%. 
Starting at the bottom, with around 10% bone, there is a design 
with a red muscle hind leg, green muscle front leg, and no bone 
in the back connecting these legs (instead there is light blue soft 
tissue). Sweeping up in that column, the percentage of bone is in¬ 
creased, predominantly in the back connecting the legs, and the 
soft tissue and amount of muscle in each leg is reduced to gradu¬ 
ally increase the amount of dark blue bone. These same creatures 
visualized from the side (rightmost column of images in Eig. 
Top) shows that the basic biped locomotion pattern is preserved 
despite these changes, going from (in the bottom) a fast, flexible 
biped that resembles a rabbit, to a slow biped creature that re¬ 
sembles a turtle with a massive, heavy shell. MAP-Elites is thus 
achieving its goal of providing the requested variation and pro¬ 
ducing a high-performing solutio n of e ach type. True to the mo¬ 
tivation of illumination algorithms EIS, finding the fastest, heavy 
shelled turtle does not preclude finding the fastest rabbit: in this 
case, they both can win the race. 

Erom these maps, one can also learn about the fitness potential 
of different regions of the feature space. Eor example, the previ¬ 
ous example showed that, holding the percentage of voxels filled 
at 75%, the more bone in a creature, the slower it is. The full map 
reveals that is generally true for almost all body sizes. The maps 
also reveal an interesting, anomalous, long, skinny island of high 
performance in the column where the percentage of voxels filled 
is roughly 7%. It turns out that column contains a variety of dif¬ 
ferent solutions that are all one voxel wide. Some quirk of the 
simulator allows these vertical sheet organisms to perform better 
than creatures that are more than one voxel wide. It might take 
hundreds or thousands of runs with traditional optimization al¬ 


gorithms to learn of this high-performing region of the space, but 
with MAP-Elites it jumps out visually in each map. Even within 
this island, we can still see smooth gradients in the desired di¬ 
mensions of variation, starting with sheets made entirely of mus¬ 
cle and transitioning to sheets made mostly of bone. Space con¬ 
straints prevent showing all of the final elites, but we consistently 
observed that one can start in nearly any location of the map and 
smoothly vary the designs found there in any direction. A second 
example map is provided (Eig.[^ Bottom) to show that these find¬ 
ings are not limited to one run of MAP-Elites, but are consistently 
found in each map: while the actual design themes are differ¬ 
ent from map to map, the fact that MAP-Elites provides smooth 
changes in these themes according to the desired dimensions of 
variations is consistent. 

6.3 Real soft robot arm 

While the previous section featured simulated soft robots, in this 
section we test whether MAP-Elites can help find controllers for 
a real, physical, soft robot arm. The physics of this arm are quite 
complicated and difficult to simulate, making it necessary to per¬ 
form all evaluations on the real robot. That limits the number 
of evaluations that can be performed, requiring a small feature 
space. This domain thus demonstrates that MAP-Elites is effec¬ 
tive even on a challenging, real-world problem with expensive 
evaluations and a small feature space. 

We built a soft robotic arm (Eig. by connecting 3 actuated 
joints (dynamixel AX-18 servos) with highly compliant tubes 
(made of flexible, washing machine drain pipes). An external 
camera tracked a red point at the end of the arm. A solution is 
a set of 3 numbers specifying the angle of each of the 3 joints. 
Specifically, each servo can move between -150 and +150 steps 
(out of the possible range for AX-18s of -512 and + 512 steps, 
which covers all 360 degrees of rotation). When the arm is fully 
extended and horizontal, the first servo from the base is at posi¬ 
tion 150, and the other two are at position 0. 

The feature space is one-dimensional: the x-value of the red 
circle at the end of the arm (in the coordinates of the image from 
the perspective of the camera). It is discretized into 64 cells. The 
performance function is to maximize the y-value of the end of the 
arm. The experiments thus attempt to discover the boundaries of 
the workspace of the robot, which is hard to compute analytically 
with a flexible robot. 

We evaluated MAP-Elites and two controls: random sampling 
and a traditional grid search algorithm. In the random sampling 
control, each solution is determined by randomly choosing an 
angle for each joint in the allowable range. The grid search al¬ 
gorithm specifies, for each joint, eight points evenly distributed 
within the range of allowable angles for that joint, and then tests 
all combinations of the possible values for each joint. We repli¬ 
cated each experiment 10 times, except for grid search, which is 
a deterministic algorithm and thus need only be run once. Each 
MAP-Elites and random sampling experiment were allocated 640 
evaluations; the grid search required 729 evaluations (9x9x9). 

The results show that all three treatments find approximately 
the correct boundary for high values of x (~600 to ~800). Our ob¬ 
servations of the robot revealed why this is a relatively easy task: 
these points can be reached by setting the angles of the first and 
second joints (counting from the base) to put the third joint on the 
ground, and only changing the angle of this third joint. Because 
of the flexibility of the links, many different combinations of an¬ 
gles for the first and second joint result in having the wrist on the 
table. 

Intermediate values of x (approximately 400-600) represent 
harder problems, because in this range there are fewer joint angle 
values that combine to reach near the maximum height. MAP- 
Elites outperforms both grid search and random sampling in this 
region. Even when they are at their best, MAP-Elites tends to out¬ 
perform these algorithms. Eor each control algorithm, there are 
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Fig. 5. MAP-Elites does much better than a traditional evolutionary algorithm (EA) and an EA with diversity (EA-i-D) at finding high-performing solutions 
throughout a feature space. Data are from the simulated, soft robot morphologies problem domain. Top: MAP-Elites significantly outperforms the 
controls in global reliability and coverage (top, p < 0.002). Bottom: Qualitatively, example maps produced by two independent runs demonstrate 
MAP-Elites’ ability to both fill cells (coverage) and reveal the fitness potential of different areas of the feature space. Note the difference in feature-space 
exploration between the MAP-Elites illumination algorithm and the optimization algorithms. 


values in this region were their performance is especially poor. 


Lower values of x represent even harder challenges. Grid 
search found only a few points below 500, and thus provides a 
less-informative (lower-resolution) picture than MAP-Elites does. 
To have good coverage of this low-x-value region (200-500), we 
would need to significantly increase the resolution (discretiza¬ 
tion) of grid search, which would require exponentially more 
evaluations. The rarity of high-performing solutions in this part 
of the feature space results in even lower performance for ran¬ 
dom sampling. MAP-Elites, in contrast, provides many high- 
performing solutions for all values of x. These data are still 
too preliminary to provide reliable statistical results, but we plot 
them to show what we know to date about MAP-Elites and the 
controls in this problem domain. 


7 Discussion and Conciusion 

This paper introduces the term "Tllumination algorithms" for the 
class of algorithms that try to find the highest-performing solu¬ 
tion at each point in a user-defined feature space. It also intro¬ 
duces a new illumination algorithm called MAP-Elites, which is 
simpler to implement and understand than previous illumination 
algorithms, namely Novelty Search -h Local CompetitiorP and 
MOL^^. Einally, the paper presents preliminary evidence show¬ 
ing that MAP-Elites tends to perform significantly better than 
control algorithms, either illumination algorithms or optimiza¬ 
tion algorithms, on three different problem domains. Because of 
the preliminary nature of the experimental data, we do not wish 
for readers at this point to conclude anything for certain yet about 
MAP-Elites' empirical performance, but in many cases the data 
suggest that MAP-Elites is a promising new illumination algo- 
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rithm that outperforms previous ones. 

Perhaps the best way to understand the benefits of illumination 
algorithms versus optimization algorithms is to view the feature 
maps from the simulated soft robot domain (Fig. |^. Optimiza¬ 
tion algorithms may return a high performing solution, but they 
do not teach us about how key features of a search space relate 
to performance. MAP-Elites and other illumination algorithms, 
in contrast, map the entire feature space to inform users about 
what is possible and the various tradeoffs between features and 
performance. Such phenotype-fitness maps, as they are known 
in biolog}P^, are interesting in their own right, and can also be 
put to practical use. For example, a recent paper showed that the 
map can provide a repertoire of different, high-performing solu¬ 
tion that can initiate search for new behaviors in case an original 
behavior no longer works (e.g. if a robot becomes damaged or 
finds itself in a new environment;^'. 

MAP-Elites is also a powerful optimization algorithm, putting 
aside its additional benefits regarding illuminating feature land¬ 
scapes. It significantly outperformed or matched control algo¬ 
rithms according to the narrow question of which algorithm 
found the single, highest-performing solution in each run. As dis¬ 
cussed above, that could be because simultaneously searching for 
a multitude of different, related stepping stones may be a much 
better way to reach any individual stepping stone than directly 
searching only for a solution to that stepping ston^. 

For a similar reason, illumination algorithms like MAP-Elites 
may help evolutionary algorithms move closer to the open-ended 
evolution seen in the natural world, which produced a tremen¬ 
dous diversity of organisms (within one run). In nature, there 
are a multitude of different niches, and a species being good in 
one niche does not preclude a different species from being good 
in another: i.e., that bears are stronger does not crowd out the 
ability for butterflies to flourish in their own wa}P^. By simul¬ 
taneously rewarding a multitude of different types of creatures, 
MAP-Elites captures some of that diversity-creating force of na¬ 
ture. One drawback to MAP-Elites, however, is that it does not 
allow the addition of new types of cells over time that did not 
exist in the original feature space. It thus, by definition, cannot 
exhibit open-ended evolution. Nature, in contrast, creates new 
niches while filling others (e.g. beavers create new types of en¬ 
vironments that other species can specialize on). Future work is 
required to explore how to create illumination algorithms that do 
not just reveal the fitness potential of a predefined feature space, 
but that report the highest-performing solutions at each point in 
an ever expanding feature space that is intelligently enlarged over 
time. 

In conclusion, illumination algorithms, which find the highest- 
performing solution at each point in a user-defined feature space, 
are valuable new tools to help us learn about complex search 
spaces. They illuminate the fitness potential of different combina¬ 
tions of features of interest, and they can also serve as powerful 
optimization algorithms. MAP-Elites represents a simple, intu¬ 
itive, new, promising illumination algorithm that can serve these 
goals. It also captures some of the diversity generating power of 
nature because it simultaneously rewards the highest-performing 
solutions in a multitude of different niches. 

8 Alternate variants of MAP-Elites 

The following are alternate ways to implement MAP-Elites. Fu¬ 
ture research is necessary to see whether, and on which types of 
problems, any of these variants is consistently better than the sim¬ 
ple, default version of MAP-Elites used in this paper. 

Possible variants of this algorithm include: 

• Storing more than one genome per feature cell to promote 
diversity 

• Biasing the choice of which cells produce offspring, such as 


biasing towards cells who have empty adjacent cells, cells 
near low-performing areas, cells near high-performing areas, 
etc. In preliminary experiments, such biases did not perform 
better than the default MAP-Elites. 

• Including crossover. Crossover may be especially effective 
when restricted to occurring between organisms nearby in 
the feature space. Doing so allows different competing con¬ 
ventions in the population (e.g. tall, skinny organisms being 
crossed over only with other tall, skinny organisms, and the 
same for short, fat organisms). One could make crossover 
only occur within a certain radius of an individual or as 
a probabilistic function of the distance between organisms 
(leading to overlapping crossover zones), or only within cer¬ 
tain regions (more akin to an island model). Note that even 
with geographically restricted crossover, offspring could still 
end up in different areas of the feature space than their par¬ 
ents (either due to the effects of mutation or crossover). 

9 Methods 

9.1 Statistics 

The statistical test for all p values is a two-tailed Mann-Whitney 
U test. 

9.2 Hierarchical, Parallelized MAP-Elites 

To first encourage a course-grained search, and then allow for in¬ 
creased granularity, we created a hierarchical version of MAP- 
Elites. It starts with larger cells and then subdivides those cells 
over time. In this hierarchical version of MAP-Elites, the sizes 
of cells shrink during search, and thus the range of differences 
in features that an organism competes with changes, although it 
is bounded to within a cell: competition is thus still restricted to 
solutions with similar features. 

To make MAP-Elites run faster on a supercomputer containing 
many networked computers, we created a batched, parallelized 
version of hierarchical MAP-Elites. It farms out batches of eval¬ 
uations to slave nodes and receives performance scores and be¬ 
havioral descriptors back from these nodes. These optimizations 
should not have any qualitative effect on the overall performance 
of the algorithm. All of the experiments in this paper were con¬ 
ducted with this hierarchical, paralleled version of MAP-Elites. 

9.3 Experimental parameters 

Retina experiments. 20 replicates for each treatment. 

The MAP-Elites parameters are as follows: 

• starting size of the map: 16 x 16 

• final size of the map: 512 x 512 

• batch size: 2,000 

• number of iterations: 10,000 

• initial batch: 20,000 

• resolution change program (4 changes): 

- iteration 0: 64 x 64 

- iteration 1250: 128 x 128 

- iteration 2500: 256 x 256 

- iteration 5000: 512 x 512 

• feature 1: connection cost (see^ 

• feature 2: network modularity (see^ 

• performance: percent answers correct on retina problerrP^ 
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Soft robots experiments. The MAP-Elites parameters are as 
follows: 

• feature 1: percentage of bones 

• feature 2: percentage of voxels filled 

• performance: covered distance 

• starting resolution: 64 x 64 

• final resolution: 128 x 128 

• batch size: 1024 

• initial batch: 4096 

• iterations: 1400 

Soft physical arm. 10 replicates for each treatment except for 
the grid search, which is deterministic and thus requires only one 
run. 

For the grid search: 

• total number of evaluations: 512 

• discretization of the parameters: 8 steps 
For the random sampling: 

• total number of evaluations: 420 

• we report the best solution found in each of the 64 cells used 
by MAP-Elites 

For MAP-Elites: 

• total number of evaluations: 420 

• feature 1: x coordinate 

• fitness: maximize height (y coordinate) 

• starting resolution: 64 

• final resolution: 64 

• batch size: 10 

• initial batch: 120 

• iterations: 30 

9.4 Quantifiable measurements of algorithm quaiity 

The notation in this section assumes a two-dimensional feature 
map {x and y), but can be generalized to any number of dimen¬ 
sions. 

9.4.1 Global reliability 

Measures how close the highest performing solution found by the 
algorithm for each cell in the map is to the highest possible per¬ 
formance for that cell, averaged over all cells in the map. Because 
we do not know the highest performance possible for each cell, 
we approximate it by setting it equal to the highest performance 
found for that cell by any run of any algorithm. Cells that have 
never been filled by any algorithm are ignored. If an algorithm in 
a run does not produce a solution in a cell, the performance for 
that algorithm for that cell is set to 0 because the algorithm found 
zero percent of that cell's potential. 

We first define Mx,y as the best solution found across all runs of 
all treatments at coordinates x,y. If A4 = mi,-- - ,m/cisa vector 
containing the final map from every run of every treatment, then 

Mx,y = max mi(x,y) 


We then define the global reliability G{m) of a map m as follows: 


G(m) 


1 m(x,y) 


where x, y G ^ ‘ ‘ ‘ Xmax] ymin’) ‘ ‘ ‘ ymax]}/ and is 

the number of non-zero entries in M (i.e. the number of unique 
cells that were filled by any run from any treatment). 


9.4.2 Precision (opt-in reliability) 


Same as global reliability, but for each run, the normalized perfor¬ 
mance is averaged only for the cells that were filled by that algo¬ 
rithm in that run. This measure addresses the following question: 
when a cell is filled, how high-performing is the solution relative 
to what is possible for that cell? 

Mathematically, the opt-in reliability, or precision, P(m) of a 
map m is: 


P{m) 


1 m{x,y) 

n(m) ^ M(x, y) 


for X, y G { [Xmin 5 ‘ ‘ ‘ 5 Xmax ] ymin ‘ ‘ ‘ ymax\ |filledm (x, y) — 1 

where filledm(a:, y) is a binary matrix that has a 1 in an (x, y) cell 
if the algorithm produced a solution in that cell and 0 otherwise, 
and where n(M) is the number of non-zero entries in M (i.e. the 
number of unique cells that were filled by any run from any treat¬ 
ment). 


9.4.3 Coverage 

For a map m produced by one run of one algorithm, we count 
the number of non-empty (i.e. filled) cells in that map and divide 
by the total number of cells that theoretically could be filled given 
the domain (i.e. for which a genome exists in the search space that 
maps to that feature-space cell). Unfortunately, we do not know 
this total number of cells that theoretically could be filled for the 
experimental domains in this paper. We approximate this number 
by counting the number of unique cells that have been filled by 
any run from any treatment. Using the notation of the previous 
two sections, this number is n(PM), where Fm — filledM- 
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Fig. 6. Example maps annotated with example organisms from different areas of the feature space. Within a map, MAP-Elites smoothly adapts 
a design theme along the desired dimensions of variation. Between maps, one can see that there is some variation between maps, both in the 
performance discovered at specific points, and in the types of solutions discovered. That said, in general each map generally paints the same overall 
picture of the performance capabilities of each region of the feature space. Note the different scale of the bottom color map. Additional example maps 
are shown in Fig.|^ Because videos do a better job of revealing the similarity and differences in these organisms, both in their body and their behavior, 
a future draft of the paper will include a video of these individuals. 
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Fig. 7. On a real, soft robot, MAP-Elites consistently finds high-performing solutions (higher y values) across the feature space (different x values) 
than controls. 
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